With the ever-increasing role of medical images in diagnosis, treatment planning, and evaluation of treatment effects, extraction of quantitative information from these images and efficient use of the results have become a necessity. In recent years, we have developed novel three-dimensional (3D) knowledge- based methods to segment brain structures from magnetic resonance images (MRI) automatically. These methods need to be optimized, fine-tuned, and compared to other methods for the segmentation of specific brain structures that may be involved in medial temporal lobe epilepsy (mTLE). Feature extraction methods also need to be developed and optimized to characterize (i.e., determine local and global multi- parametric intensity distribution, texture, shape, surface area, surface curvature, and volume of) the brain structures. Multi-modality analysis using multi-parametric MRI and SPECT needs to be developed for improved sensitivity and specificity. We have also developed our preliminary version of a content-based human brain image database system to hold the image analysis results with other clinical information (e.g., textual data) in a manner that can be searched, retrieved, and queried conveniently from any computer station. This system needs integrated methods for data preparation, missing value treatment, interactive rule-extraction, visualization, and user-inference to serve as a decision support system in clinical practice. A user-friendly, web-based interface will be critical for the ultimate use of the system by researchers and clinicians. Last but not least, the database needs to be populated with data from a large number of patients so that it can be confidently used for hypothesis testing and clinical applications. The goal of this project is to develop novel approaches for the above needs. Image analysis and feature extraction methods will segment and characterize hippocampus, amygdala, entorhinal cortex, thalamus, putamen, and other brain structures from MRI. The methods will be tested, evaluated, and validated using clinical data of epilepsy patients. Clinical diagnosis based on EEG studies and surgery outcome will be used as gold standards for evaluation and validation of the image analysis methods. The proposed decision support system will be populated with multi-modality data of 350 epilepsy patients to evaluate correlation between a variety of risk factors, imaging features, clinical diagnosis (lateralization), and post- operative outcomes, and to assist physicians with improved clinical diagnosis, reduced intracranial EEG studies (reduced risk and suffering of patients as well as their healthcare cost), optimal treatment options, and prediction of outcome in prospective studies. The proposed research will be a breakthrough in the application of computerized methods for medical image quantification and object characterization, and will advance image analysis science in the direction of integrating knowledge-based image segmentation and characterization methods with pattern recognition and data mining technology in decision support systems. The proposed approaches are applicable to the identification, segmentation, and characterization of other biological structures. They are also applicable to virtually any image analysis task for which object segmentation, quantification, and characterization are used.